Skip to content

Conversation

@DeagleGross
Copy link
Member

@DeagleGross DeagleGross commented Jul 31, 2025

Description

PR is an implementation of WithExecCommand extension (see In Aspire.Hosting in #10301), which allows to register a command to execute against a container resource (actually inside the container - for example ls (list files) in the nginx container as is shown in tests). WithExecCommand is similar to WithCommand because it annotates a container resource with the new ResourceContainerExecCommandAnnotation. It is very similar to existing ResourceCommandAnnotation.

There has to be a way to actually launch a command and it will be implemented in the follow-up PR for #10301, part In the dashboard), and new ContainerExecService exposes an API to do it. WithCommand has different semantics: since the "command" is basically a c# callback, we can know if execution of the "command" has completed successfully, and we can return it. In case of containers there is no reliable way to know how the command execution went, and the most important here is to fetch logs, so I've built API which returns an IAsyncEnumerable<ContainerExecCommandOutput> - basically a stream of logs.

On the opposite of the WithCommand here it is necessary to run the the ContainerExec inside of DCP to actually execute the command inside the container. See implementation details below.

Public API changes

namespace Aspire.Hosting.ApplicationModel;

+ public class ContainerExecCommandOutput
+ {
+   public required string Text { get; init; }
+   public required bool IsErrorMessage { get; init; }
+   public int? LineNumber { get; init; }
+ }

+ public class ResourceContainerExecCommandAnnotation : ResourceCommandAnnotationBase
+ {
+   public string Command { get; }
+   public string? WorkingDirectory { get; }
+ }
namespace Aspire.Hosting.Exec;

+ public class ContainerExecService
+ {
+   public async IAsyncEnumerable<ContainerExecCommandOutput> ExecuteCommandAsync(string resourceId, string commandName, [EnumeratorCancellation] CancellationToken cancellationToken = default);

+   public IAsyncEnumerable<ContainerExecCommandOutput> ExecuteCommandAsync(ContainerResource resource, string commandName, CancellationToken cancellationToken = default);
+ }

Implementation

I decided to allow IDcpExecutor to do 2 operations:

interface IDcpExecutor
{
+    Task<AppResource> RunEphemeralResourceAsync(IResource ephemeralResource, CancellationToken cancellationToken);
+    void DeleteEphemeralResource(AppResource? ephemeralResource);
}

The idea is that we dont know what custom commands will be executed against the container. In #10301 we will also support running custom commands from the dashboard via the interaction with the user. So we not always have an opportunity to register the ContainerExecutableResource before DCP starts, so I've extended the possibilities of DcpExecutor to run a new resource and to delete the handlers to it (only for monitoring purposes). I know DcpExecutor was an immutable entity before, so this can be a concerning change.

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
      • If yes, have you done a threat model and had a security review?
        • Yes
        • No
    • No
  • Does the change require an update in our Aspire docs?
    • Yes, will add later once full feature is complete

Copilot AI review requested due to automatic review settings July 31, 2025 13:25
@DeagleGross DeagleGross requested a review from mitchdenny as a code owner July 31, 2025 13:25
@github-actions github-actions bot added the area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication label Jul 31, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new extension API WithExecCommand that allows registering commands to execute inside container resources. The feature enables executing commands like ls directly within containers and streaming the output back as logs, similar to existing WithCommand but specifically designed for container execution scenarios.

Key changes:

  • New WithExecCommand extension API for registering container execution commands
  • ContainerExecService for executing commands and streaming output from containers
  • Extended IDcpExecutor interface to support ephemeral resource execution and cleanup

Reviewed Changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
src/Aspire.Hosting/ResourceBuilderExtensions.cs Adds the new WithExecCommand extension method
src/Aspire.Hosting/ApplicationModel/ResourceContainerExecCommandAnnotation.cs New annotation class for container exec commands and output model
src/Aspire.Hosting/ApplicationModel/ResourceCommandAnnotationBase.cs New base class extracting common command annotation functionality
src/Aspire.Hosting/ApplicationModel/ResourceCommandAnnotation.cs Refactored to inherit from new base class
src/Aspire.Hosting/Exec/ContainerExecService.cs New service for executing container commands and streaming output
src/Aspire.Hosting/Dcp/IDcpExecutor.cs Extended interface to support ephemeral resource operations
src/Aspire.Hosting/Dcp/DcpExecutor.cs Implementation of ephemeral resource execution and cleanup
src/Aspire.Hosting/Dcp/DcpResourceState.cs Added methods for managing ephemeral resources in state
src/Aspire.Hosting/DistributedApplicationBuilder.cs Registered ContainerExecService in DI container
Various backchannel files Renamed CommandOutput to BackchannelCommandOutput for clarity
Test files Added comprehensive tests for the new functionality

/// <param name="command">The command string to be executed.</param>
/// <param name="commandOptions">Optional settings for the command, such as description and icon.</param>
/// <returns>The resource builder, allowing for method chaining.</returns>
public static IResourceBuilder<T> WithExecCommand<T>(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about ExecCommand as the name.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as per requirement in the issue (see #10301), it should be with the name.
It allows to add multiple commands per name to the same container resource. for example

builder
  .AddResource(containerResource)
  .WithExecCommand("listfiles", ...)
  .WithExecCommand("deletefiles", ...)

Why are you not sure about such API? It is basically a copy-cat from WithCommand

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open to suggestions RE naming. This follows the pattern established by WithHttpCommand.

@karolz-ms
Copy link
Member

(from the PR description)

In case of containers there is no reliable way to know how the command execution went, and the most important here is to fetch logs, so I've built API which returns an IAsyncEnumerable - basically a stream of logs.

Not sure what you mean. The status for ContainerExec object has a State property and an ExitCode property https://github.com/dotnet/aspire/blob/main/src/Aspire.Hosting/Dcp/Model/ContainerExec.cs#L56 so you can know extactly what the outcome of an container executable was, once it has completed.

Copy link
Member

@karolz-ms karolz-ms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main concern is the interface for executing a container command, it needs to be re-designed IMO.


if (ephemeralResource.ModelResource is ContainerExecutableResource)
{
_resourceState.Remove(ephemeralResource);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the resource be absent from either of these two lists? If so, what should happen?

Is it enough to just remove the resource from these two lists without doing any additional cleanup? Like, stopping log streams associated with the resource? If the answer is "yes, it is sufficient, no extra cleanup is needed", then at least it is worth putting a comment here explaining WHY

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've included a comment on the collection addition/removal. Those collections are a part of DcpExecutor to enable correct monitoring of resource state and log collection. It is an obscure part of DcpExecutor and it needs a refactor to have more convenient API.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is that comment? I do not see any in the latest version of DeleteEphemeralResourceAsync() method?

Copy link
Member

@karolz-ms karolz-ms Aug 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are RunEphemeralResourceAsync() and DeleteEphemeralResourceAsync() methods tread-safe? (I assume not) What are the assumptions we are making about the context from which they will be called?

@dotnet-policy-service dotnet-policy-service bot added the needs-author-action An issue or pull request that requires more info or actions from the author. label Jul 31, 2025
@mitchdenny
Copy link
Member

OK, so I've reviewed this code. I definitely do not want us to introduce ResourceCommandAnnotationBase. I see this as unnecessary. I think the reason it feels necessary is that we've got an inverted relationship between ResourceContainerExecCommandAnnotation and ContainerExecService.

What we should be doing is in the WithExecCommand(...) method we should be invoking WithCommand(...) and into the callback we should be making a call to the DI container to resolve the ContainerExecService (or its interface) and then using that to invoke the command in the selected container resources. The parameters that are used to invoke the API on ContainerExecService would be passed in to the WithExecCommand(...) and captured in the lambda that is passed into the WithCommand(...) extension method.

We should look to WithHttpCommand(...) for inspiration here. the WithExecCommand(...) should probably have the essential arguments that we need for invoking a command inside the container - plus a derived ExecCommandOptions type.

One thing that we should probably think carefully about here is whether it actually makes sense to define container exec commands in the app model as a new kind of resource. Presently we have ContainerResources - perhaps we should have a ContainerExecutableResource which derives from ExecutableResource.

This way we might be able to take advantage of all of the plumbing we already have in Aspire around handling log streaming etc.

I believe that aspire exec will need to be converted with aspire run and that when doing aspire exec we will need to show the dashboard. We should probably resolve this before going too much further.

@mitchdenny
Copy link
Member

@DeagleGross please look to address some of the feedback I've added as comments to this commit in this PR:

#10380 (comment)_

#10380 (comment)_

If we are introducing a resource into the app model and we implement interfaces like IResourceWithArgs we need to make sure that we honor the semantics of those interfaces. That means things like the Args property on the ContainerExecutableResource is redundant because we capture arguments in a series of callback arguments which get set with things like WithArgs(...).

Additionally instead of having a TargetResource property we should just implement IResourceWithParent I think.

@DeagleGross DeagleGross self-assigned this Aug 1, 2025
@dotnet-policy-service dotnet-policy-service bot removed the needs-author-action An issue or pull request that requires more info or actions from the author. label Aug 1, 2025
@DeagleGross
Copy link
Member Author

@mitchdenny thanks for the explanation about similarities with WithCommand, but I disagree - the API is simply different. In case of container exec underlying execution happens in a separate DCP resource, and we should have all the infra working for us providing a way to fetch logs for example. Trying to pack everything into WithCommand with its hardly binded Func<ExecuteCommandContext, Task<ExecuteCommandResult>> contract is not extensible enough. Please see the updated code and the idea by Karol here: #10779 (comment).

However, I've refactored the code and removed the base annotation class etc, so its cleaner now.

The cli changes for aspire exec to be based on top of aspire run is unrelated to this flow, and should come in a separate PR. I will also definitely fix what you have described in other PR (#10380 (comment)_
#10380 (comment)_); but will do in a separate PR for simplicity, if you are fine with that - this PR should not make it harder, because it provides a very simple command definition and args are not touched here


Func<CancellationToken, Task<ExecuteCommandResult>> commandResultTask = async (CancellationToken cancellationToken) =>
{
await _dcpExecutor.RunEphemeralResourceAsync(containerExecResource, cancellationToken).ConfigureAwait(false);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems awkward that the call to ContainerExecService.ExecuteCommand() does not, in fact, executes the command.

Consider making this method an async method and taking the call to RunEphemenralResourceAsync() out of the result-producing task. This will guarantee that when ExecuteCommandCore() returns without exception, the command is started. The returned run object would then expose GetResultAsync() method for waiting on the result.

This change also guarantees that calling GetOutputStream() on the returned run object always makes sense. Without it, it is possible to ask for output stream without actually starting the command, which will result in waiting forever (until the cancellation token expires)

@dotnet-policy-service dotnet-policy-service bot added the needs-author-action An issue or pull request that requires more info or actions from the author. label Aug 1, 2025
@mitchdenny
Copy link
Member

@mitchdenny thanks for the explanation about similarities with WithCommand, but I disagree

I think there might be a bit of a disconnect here. When I look at WithExecCommand(...) I initially thought of it as being analogous to WithHttpCommand(...). But in reality, WithExecCommand(...) is much more like a child resource of a ContainerResource.

With that in mind I'm wondering if we are conflating two things here - commanding on the dashboard, and the introduction of a sub-resource for containers capable of executing resources within a running container.

If we imagine that dashboard commands don't exist for a moment, the way we might add this feature is that we would do something like this:

var builder = DistributedApplication.CreateBuilder(args);
builder.AddContainer("mycontainer", "myimage")
       .AddInvocation("truncate-database", "dbtool", "truncate");

Note I'm using AddInvocation(...) here as a stand-in for *Command* or *Exec* because I think that the usage is a bit overloaded.

What the code above would do is that the container would start, and there would be an implicit WaitFor(...) on the parent resource before the truncate-database resource executed.

Now - the interesting here is how this maps onto aspire exec. For aspire exec you wouldn't have the call to AddInvocation(...) instead you would probably just use --resource to identify the parent container resource - and if it is a container resource instead of adding a ExecutableResource add a ContainerExecutableResource.

This doesn't really have anything to do with dashboard commands - in fact they aren't even really required. Because ContainerExecutableResource is a resource and has lifecycle and logs and all that good stuff - it would be visble in the dashboard. If you wanted to define a ContainerExecutableResource using something like AddInvocation(...) then you could but you might also append WithExplicitStart(...) to stop it starting automatically - and when you wanted to run it you would just click start in the dashboard.

You could - if you wanted to also hide the resource so that it is not alway visible and then add a command using WithCommand(...) (assuming you are building an integration that is layered on top of all this).

That WithCommand(...) would use the ResourceCommandService to simply start the container executable resource just like any other resource and potentially make the resource visible in the dashboard and present an interaction dialog which jumps the user to the logs.

@karolz-ms
Copy link
Member

@mitchdenny and @DeagleGross, just wanted to point out that ContainerExec in DCP represents a single invocation of a command within a container. It is not a "template" for command execution. ContainerExec objects can be created at any time during the corresponding container lifetime.

I guess how we incorporate this capability into Aspire model is up for debate. We need to consider deployment environments which may or may not have that capability for containers. Personally, I am not very sure if modeling "execution of programs within containers" (avoiding the overloaded term "command" here) is a good candidate for being a distinct resource type in Aspire model. I am wondering if something more lightweight (e.g. just a method on the container resource) would ultimately be easier to use and more appropriate.

@DamianEdwards
Copy link
Member

Personally, I am not very sure if modeling "execution of programs within containers" (avoiding the overloaded term "command" here) is a good candidate for being a distinct resource type in Aspire model. I am wondering if something more lightweight (e.g. just a method on the container resource) would ultimately be easier to use and more appropriate.

This is exactly my thinking too. I do not see this as a resource at all, but rather exposing the ability to execute/invoke something in the context of a resource.

@mitchdenny
Copy link
Member

My concern is that if we don't model this as a resource from top to bottom then we are going to end up with unnecessary duplications for something that conceptually is very well aligned with resources in the Aspire app model.

I mean we define models in Ollama as resources - why not a docker exec invocation? We've found that even for resources that aren't executables that there is a need to be able to inspect the output that they produce - it just makes sense that we would use the existing dashboard infrastructure for this doesn't it?

@dotnet-policy-service
Copy link

This submission has been automatically marked as stale because it has been marked as requiring author action but has not had any activity for 14 days.
It will be closed if no further activity occurs within 7 days of this comment.

@github-actions github-actions bot locked and limited conversation to collaborators Sep 26, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication needs-author-action An issue or pull request that requires more info or actions from the author. no-recent-activity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants